Method of 3d reconstruction and 3d panoramic mosaicing of a scene

ABSTRACT

The invention relates to a method of 3D reconstruction of a scene by means of 2D panoramic images of the scene, which comprises a step of processing these images. These images arise from a panoramic system moving in displacement along a determined trajectory, such that the image of at least one point of the scene is in at least 3 successive images obtained according to various directions; the step of processing these 2D successive images comprises the sub-steps:
         a) determining reconstruction planes in the scene to be reconstructed,   b) determining, on the basis of pairs of panoramic images and for each pair, rectification planes corresponding to the reconstruction planes and projecting onto each of them a sector of each image of the pair, in a direct manner, so as to obtain two 2D rectified images,   c) matching the two 2D rectified images so as to obtain an intermediate 3D reconstruction,   d) transforming each intermediate 3D reconstruction into a 3D frame including the reconstruction planes so as to obtain a transformed intermediate 3D reconstruction,   e) repeating steps b) to d) on the basis of a new pair of 2D panoramic images and of at least one other rectification plane, so as to obtain at least one other transformed intermediate 3D reconstruction,   f) temporally fusing at least two transformed intermediate 3D reconstructions so as to obtain a 3D reconstruction of the scene.

The field of the invention is that of the 3D reconstruction of a sceneon the basis of successive panoramic images of this scene, optionallyfollowed by the 3D mosaicing of this scene.

The 3D reconstruction of a scene consists in obtaining, on the basis ofsuccessive 2D images of this scene taken from different viewpoints, aso-called 3D reconstructed image such that with each pixel of thereconstructed image, that is to say at any point where thereconstruction declares that there is a scene element, are associatedthe coordinates of the point of the corresponding scene, defined in aframe X, Y, Z related to this scene.

Conventional mosaicing, so-called 2D mosaicing, consists on the basis ofsuccessive images of a scene in projecting them successively onto aprincipal plane of the scene and in assembling them to produce a mosaicthereof.

Techniques for passive 3D scene reconstruction on the basis of camerasare described in various reference works:

-   -   R. Horaud & O. Monga. Vision par Ordinateur: Outils        Fondamentaux, Editions Hermés, 1995.        http://www.inrialpes.fr/movi/people/Horaud/livre-hermes.html    -   Olivier Faugeras. Three-Dimensional Computer Vision, MIT Press,        1993    -   Frédéric Devernay, INRIA Grenoble, course “Vision par ordinateur        3-D”. http://devernay.free.fr/cours/vision/    -   Tébourbi Riadh, SUP′COM 2005 IMAGERIE 3D 08/10/2007    -   “Learning OpenCV: Computer Vision with the OpenCV Library”, Gary        Bradsky, 2008

These works all cite techniques for 3D scene reconstruction on the basisof pairs of stereoscopic images originating from cameras positioned atdifferent viewpoints, which may either be fixed cameras positioned atvarious sites in space, or a camera whose position varies temporally,always with the same basic principle of matching the images of thecameras taken 2 by 2 to form a stereoscopic 3D reconstruction of theportion of space viewed by the cameras.

They also explain the principle of epipolar rectification where thefocal plane image of each camera is rectified according to the attitudeof the camera on a so-called rectification plane so as to facilitate thematching between the images of the stereoscopic pair and enable the 3Dreconstruction. The method is relatively optimized by various authorsbut always relies on the principle that it is firstly necessary tocorrect the optical distortions of the camera and thereafter to use therelative attitudes of the 2 cameras to determine the rectification planeon the basis of which the matching and the 3D reconstruction areperformed.

Other techniques of passive 3D reconstruction exist in the literature,for example the so-called silhouetting techniques, not considered heresince they apply to particular cases and require prior knowledge aboutthe scene.

In the techniques of active reconstruction of a scene, it is possible tocite those based on lidar which make it possible to reconstruct the 3Dmesh of the scene directly by a distance computation.

Among the reference works may be cited:

-   -   MATIS studies for the IGN: “Using Full Waveform Lidar Data for        Mapping of urban Areas”, Doctoral thesis, Clément Mallet, 2010    -   “Couplage de Données Laser Aéroporté et Photogrammétriques pour        l'Analyse de Scénes Tridimensionnelles”, Doctoral thesis,        Frédéric Bretar, 2006.

An interesting article shows that these techniques have limits inreconstructing 3D objects of complex shape (for example concave):Structuration plane d'un nuage de points 3D non structuré et détectiondes zones d'obstacles, Vision interface conference, 1999, NicolasLoémie, Laurent Gallo, Nicole Cambou, Georges Stamon.

Concerning mosaicing, the following reference works may be cited:

-   -   L. G. Brown, “A Survey of Image Registration Techniques”, in ACM        Computing Surveys, vol. 24, n° 4, 1992,    -   “Mosaïque d'images multiresolution et applications”, Doctoral        thesis, Université de Lyon. Lionel Robinault, 2009.

If one summarizes the prior art relating to 3D reconstruction, it may besaid that 3D reconstruction may be partially obtained by using:

-   -   Pairs of cameras producing a spatially stereoscopic image of the        scene and by fusing these images to produce a 3D reconstruction        and optionally a mosaicing of the scene. This solution exhibits        several drawbacks:        -   the cameras are difficult to calibrate (problems of            vibration),        -   an inaccuracy in restitution of the 3D reconstruction on            account of a stereo base limited by the spacing between the            cameras,        -   low-field and low-extent restitution on account of the            limited optical field of the cameras.

Moreover, the finalized 3D reconstruction is not obvious, since it isconstructed by assembling local 3D reconstructions (resulting from themethod of stereoscopic restitution of 2, often small-field, images)which may be very noisy on account of the limited number of images whichmade it possible to construct it, of the limited field of the camerasand of the fact that the reconstruction planes dependent on therespective attitudes of the cameras have a geometry that is difficult tomeasure accurately (the relative position and relative geometry of thecameras serving to do the 3D reconstruction is often inaccurate inpractice when dealing with cameras which are 1 or 2 meters apart andliable to vibrate with respect to one another: this is still moreevident when these cameras are motorized). The precise way of assemblingthe intermediate 3D reconstructions is never described in detail and inpractice many errors are noted in the finalized 3D reconstruction whichin any event remains small in spatial and angular extent (typically lessthan 200 m×200 m in spatial extent with an angular extent of typicallyless than 30°).

Finally the rectification and matching method itself, dependent on theattitudes of the cameras and entailing a preliminary step of derotationof the focal plane in the rectification process, implies that typicalcases exist where the 3D reconstruction exhibits holes, especially ifthe system exhibits temporal rotation motions.

Lastly, the stereoscopic system restores poorly planes which are almostperpendicular to one of the 2 cameras (this is the problem of therestitution of pitched roofs in aerial or satellite stereoscopicimaging).

A moving low-field or mean-field camera, but the 3D reconstruction islimited by the path and the orientation of the camera and is thereforenot omnidirectional; moreover, the reconstruction may exhibit holes onaccount of unchecked motions of the camera or non-overlaps of the latterin the course of its motion. The algorithms used for 3D reconstructionimpose a reconstruction in a frame tied or close to the focal plane ofthe camera, thereby limiting the possibilities of reconstruction (asingle principal reconstruction plane and very limited reconstructionwhen the camera changes orientation). The result of the reconstructionis also very noisy and may exhibit numerous errors on account of thesmall overlap between images, of a constant plane of reconstruction ofthe reconstructed scene (and of a camera that could deviate from thisplane) and of the use of algorithms which for the 3D reconstructionutilize only two images separated by a relatively small distance. Themosaicing obtained by the ground overlaying of the successive images isinoperative and is not conformal when the scene is not flat and/orcomprises 3D elements.

Active sensors, that is to say with telemetry, but here again the 3Dreconstruction is not omnidirectional and is not necessarily segmented,the measurements being obtained in the form of scatters of points thatare difficult to utilize in an automatic manner. Moreover, the meshobtained by these active sensors exhibits the drawback of beingangularly non-dense (typically fewer than 4 points per m² for airborneapplications at 1 km height). The technique is not at the momentsuitable for being able to produce a textured image of the scene andmust almost always be corrected manually.

All the previous solutions are unsuitable for obtaining a 3D mosaicingor a 3D reconstruction for a 3D scene of large dimension, that is to saygreater than 500 m×500 m. The 3D instantaneous mosaics obtained exhibitdeformations and are limited in angular extent (typically <30°) orspatial extent. The assembling of the mosaics is complex when theterrain is 3D and the final result does not conform to the geometry ofthe scene.

The drawbacks of the procedures of the prior art are not limiting; otherdrawbacks are described in the patent.

The aim of the invention is to alleviate these drawbacks.

The proposed solution is based on the use of a panoramic system coveringa very large angular sector of the scene and able if so desired to be aslarge as to cover the complete sphere, and on the utilization of thedisplacement of the sector in the scene with a processing of the 2Dimages obtained, so as to reconstruct the 3D information of the scene inthe entirety of what has been viewed by the panoramic system andaccording to various reconstruction planes.

The panoramic system comprises one or more sensors whose images do notnecessarily exhibit any mutual overlap, and makes it possible to coverthe whole of the scene to be reconstructed instantaneously (with holesif the sensors are not overlapping) or in the course of the motion.

This solution also makes it possible to produce in parallel a mosaicwith very large spatial and angular extent representing this sceneaccording to all the viewpoints of the panoramic system in the course ofits displacement.

More precisely, the subject of the invention is a method of 3Dreconstruction of a scene by means of 2D panoramic images of the scene,which comprises a step of processing these 2D panoramic images. It isprincipally characterized in that the 2D panoramic images arise from apanoramic system moving in displacement along a determined trajectory,such that the image of at least one point of the scene is in at least 3successive 2D panoramic images obtained according to various panoramicsystem-point of the scene directions, and in that the step of processingthese 2D successive panoramic images respectively obtained at successiveinstants comprises the following sub-steps:

-   -   a) determining reconstruction planes in the scene to be        reconstructed,    -   b) determining, on the basis of panoramic image pairs        respectively formed of a 2D panoramic image obtained at an        instant t and of another 2D panoramic image obtained at an        instant t+Δt and for each chosen pair of images, rectification        planes corresponding to the various reconstruction planes chosen        and projecting onto each of them a sector of a 2D panoramic        image of the pair, in a direct manner, that is to say without        passing through one or more intermediate planes dependent on the        line of sight of the panoramic system, so as to obtain a 2D        rectified image, and projecting onto these same rectification        plane and in a direct manner a sector of the other 2D panoramic        image of the pair so as to obtain another 2D rectified image,    -   c) matching, for each chosen rectification plane, the two 2D        rectified images so as to obtain an intermediate 3D        reconstruction in a 3D frame tied to the rectification plane,        the so-called intermediate 3D frame,    -   d) transforming each intermediate 3D reconstruction into a 3D        frame including the reconstruction planes chosen in a), the        so-called 3D reconstruction frame, so as to obtain a transformed        intermediate 3D reconstruction,    -   e) repeating steps b) to d) at least once on the basis of a new        pair of 2D panoramic images and of at least one other        rectification plane, so as to obtain at least one other        transformed intermediate 3D reconstruction in this 3D        reconstruction frame,    -   f) temporally fusing, for each chosen construction plane, at        least two transformed intermediate 3D reconstructions so as to        obtain a 3D reconstruction of the scene,    -   g) steps b) to f) are carried out for each reconstruction plane        chosen in step a), with the same panoramic images but with        different sectors, so as to thus obtain as many 3D        reconstructions as reconstruction planes chosen.

This method makes it possible to find the most appropriate dense 3D meshto represent the scene, such that with each point of this mesh areassociated the coordinates of the corresponding point in a frame X,Y,Ztied to the scene.

The repetitions make it possible to have a continuous process forupdating the 3D models.

These 3D reconstructions or the intermediate 3D reconstructions canadvantageously be fused together spatially so as to update the final 3Dreconstruction or reconstructions.

Δt is typically determined as a function of the desired accuracy for the3D reconstruction.

Each rectification plane is determined as a function of the trajectoryof the panoramic system, and is independent of the line of sight of thepanoramic system.

This trajectory may be arbitrary and/or determined in tandem with theprogress of the 3D reconstruction method.

Each reconstruction plane in the scene is determined experimentally bythe operator or automatically as a function of the scene alreadyrestored; it can also be determined as a function of the trajectory ofthe panoramic system, and is independent of the line of sight of thepanoramic system.

According to a characteristic of the invention, the panoramic systemexhibiting a determined field of vision, the repetition in step e) isperformed as long as the point of the scene is in the field of vision ofthe panoramic system.

The subject of the invention is also a method of mosaicing of a scene,characterized in that it comprises 3D reconstruction steps such asdescribed above and in that textures are associated with each 3Dreconstruction, and in that it furthermore comprises the followingsteps:

-   -   A) Choosing one or more projection surfaces on which the mosaic        will be constructed,    -   B) If rectified images arising from the 3D reconstruction can be        determined, determining them and extracting in these rectified        images visible parts with the aid of the 3D reconstruction, and        selecting corresponding textures,        -   Otherwise:            -   acquiring a 2D panoramic image, the so-called current                panoramic image,            -   determining sectors of the current panoramic image,            -   extracting in these sectors visible parts with the aid                of the 3D reconstruction, and selecting corresponding                textures,    -   C) Projecting these textures onto each projection surface and        fusing the textures in each projection surface so as to thus        obtain a conformal mosaic on each projection surface.

These steps are preferably repeated at each new 2D panoramic imageacquisition.

This therefore produces a 3D mosaicing which is a generalization of 2Dmosaicing in the sense that the projection can be done on an arbitrary3D surface, which may itself consist of several plane or non-planesurfaces exhibiting discontinuities. This 3D mosaicing consists, on thebasis of successive 2D images of a scene (taken from differentviewpoints) and of the 3D reconstruction of the scene in the abovesense, in projecting and assembling the various 2D images on thegeometric modeling of the 3D reconstruction, thus making it possible torestore the whole of the scene in the form of a textured mosaic overlaidon the various 3D elements of this scene. It makes it possible torestore in a conformal manner an assemblage of images on an arbitraryscene exhibiting relief or 3D elements. The reconstructed 3D mosaic istherefore a textured 3D reconstruction of the scene.

These methods make it possible to carry out a 3D reconstruction and a 3Dmosaicing over the widest possible spatial and angular extent.

The invention also relates to an apparatus for 3D reconstruction of ascene, which comprises:

-   -   a panoramic system able to form 2D images of the scene,        so-called 2D panoramic images, and furnished with location means        and,    -   linked to this panoramic system, a computer comprising:        -   means for implementing the method of 3D reconstruction and            optionally of mosaicing as defined above,        -   automatic complementary image processing means optionally            associated with a man-machine interface or replaced with it.

A simple, accurate method is thus obtained making it possible forexample to produce textured maps on which measurements can be performed,to reconstruct the scene over a large spatial (possibly up to as much as180°×360° and angular extent and in real time, without constraints onthe trajectory, on an arbitrary scene (without any plane-sceneassumptions and without the aid of any prior scene model for example).

The proposed solution:

-   -   makes it possible to afford a compact solution to the problem of        3D reconstruction and/or of mosaicing of the scene by requiring        only a single panoramic system, whilst those of the prior art        require several independent sensors and are more complex to        implement,    -   produces a 3D reconstruction and optionally a mosaic of the        scene        -   which are conformal, that is to say without geometric            deformations and therefore superimposable on a map,        -   over a very wide spatial and angular extent, and without            holes,        -   which are complete, that is to say that can be done            according to planes of various directions, this being very            different from the conventional methods producing only a            single reconstruction plane as output, and not making it            possible to restore or restoring poorly objects of the scene            having faces different from the chosen restitution plane,        -   which are robust by virtue of the temporal redundancies            implemented,        -   which are accurate by virtue of the temporal stereovision            which on its own produces a virtual stereoscopic base of            large dimension, thereby explaining the accuracy,        -   which are instantaneous, in the sense that the 3D panoramic            restitution and the associated mosaic are recomputed and            updated at each instant,        -   which are compatible for example with bottom of the range            MEMS attitude platforms when the latter are used to            ascertain the trajectory, or with simple means of            measurement of relative displacements such as an odometer or            a basic GPS,        -   which are compatible with ample or uncoordinated motions of            the sensor, something that a small-field sensor does not            allow,        -   and which applies to any type of trajectory, including            curvilinear and in any direction.

Other advantages may be cited, such as:

-   -   allows the operator to choose arbitrary reconstruction planes        (so as for example to reproduce at one and the same time what is        on the ground and on the facades, or according to a cylindrical        projection). The solution is also suitable for the reproduction        of concave objects, this being very difficult to carry out by        other procedures,    -   optionally produces textured reconstructions whereon accurate        measurements are possible (the reproduced images are conformal),    -   allows arbitrary motions of the panoramic system in the scene,        including coming closer,    -   does not require any external measurement other than those        measuring the relative displacement in position and attitude        between 2 image shots, with a measurement accuracy compatible        with bottom of the range COTS instruments (MEMS platform, basic        GPS or odometer,).    -   does not require any other a priori information about the scene        to be reconstructed,    -   allows real-time utilization on a PC.

Other characteristics and advantages of the invention will becomeapparent on reading the detailed description which follows, given by wayof nonlimiting example and with reference to the appended drawings inwhich:

FIG. 1 schematically represents an exemplary apparatus for theimplementation of the method of 3D reconstruction and optionally ofmosaicing according to the invention,

FIG. 2 schematically represents various steps of the 3D reconstructionmethod according to the invention,

FIG. 3 schematically represents various steps of the method of mosaicingaccording to the invention,

FIG. 4 illustrates measurement ambiguities produced by a concave objectwhen there is only a single reconstruction plane,

FIG. 5 represents an exemplary sectorial decomposition of a panoramicimage resulting from a panoramic system,

FIG. 6 represents examples of rectified images of sectors of thepanoramic image of FIG. 5, projected onto various rectification planes,

FIG. 7 schematically represents, for an exemplary trajectory, examplesof rectification planes, lines of sight of the panoramic system Ldv1 andLdv2 being independent of these planes,

FIG. 8 schematically represents an exemplary temporal evolution ofrectification planes and of 3D reconstruction planes according to theinvention, for a given trajectory.

The general idea of the invention consists in utilizing to the maximumthe frontal angular field (frontal=the direction of whose line of sighttraverses a plane in the direction of the motion of the panoramicsystem) and transverse angular field (transverse=the direction of whoseline of sight traverses a plane in the direction perpendicular to themotion of the panoramic system) of a panoramic system moving in a sceneaccording to a known trajectory, to restore, according to variousviewpoints, the relief and optionally the texture of this scene.

The utilization of the transverse field is done by reconstructing therelief and optionally the texture according to all the lateralviewpoints viewed by the panoramic system that can be presented to theoperator according to various reconstruction planes.

The utilization of the frontal field is done by utilizing the temporalfusion of the previous reconstructions observing the objects of thescene according to different viewpoints. These various reconstructionsof an object viewed according to various viewpoints make it possible toproduce an extended, accurate and conformal global view of the scenewhich can be exhibited to an operator according to various viewpoints.

Utilizing temporal stereoscopy in various angular directions that can beproduced by the displacement of the panoramic optical (or optronic)system moving in a scene makes it possible to simultaneously produce a3D reconstruction of the scene projectable according to variousdirections and optionally a conformal and multi-face mosaic of thescene.

The proposed solution uses the following new concepts:

-   -   temporal stereoscopy with panoramic system, which is        differentiated from conventional stereoscopy using two        small-field cameras,    -   simultaneous rectification according to various planes whose        directions are chosen freely, which is differentiated from        conventional rectification which is done only on a single plane        whose direction is imposed by the direction of line of sight of        the two sensors used. Another innovation is direct rectification        which is done directly between any part of the 2D image of the        panoramic system and the chosen rectification plane, in        contradistinction to conventional rectification used in        stereovision which imposes an intermediate straightening plane,        thereby producing losses of information,    -   fusion of intermediate reconstructions utilizing very different        directions of line of sight, making it possible to gain accuracy        and robustness,    -   confidence map related to the hierarchization of the quality of        the information extracted from the 2D images aimed at an object        of the scene on very different viewpoints and which is directly        related to the temporal utilization of the 2D images of a        panoramic system in motion.

More precisely, the method is implemented by means of an apparatus anexample of which is shown in FIG. 1, which comprises:

-   -   a panoramic system 1 able to form 2D panoramic images of the        scene, comprising a sensor 14 associated with an optic 11 and        furnished with location means such as a GPS 12 and an inertial        platform 13, and,    -   linked to this panoramic system, a computer 2 comprising:        -   means 21 for implementing the method of 3D reconstruction            and optionally of 3D mosaicing such as described, and        -   automatic complementary image processing means optionally            associated with, or replaced with, a man-machine interface            22.

According to the invention, the 2D images arise from the panoramicsystem 1 moving in displacement along a known trajectory, the latterbeing able to be measured in relative from image to image in tandem withthe displacement, by virtue of the location means and the computer 2.

The system is panoramic in the sense that it makes it possible to obtaina 2D panoramic image. For this purpose, it can comprise a large-fieldoptic 11 of fisheye type, or any conventional or catadioptriclarge-field optical means able to provide a 2D panoramic image, or elseon the basis of a smaller-field optic but which moves with more or lessample motions so as to sense the various portions of scenes that it isdesired to reconstruct in their entirety. A 2D image covering a largefield of greater than 60° is for example obtained on the basis of asystem 1 with 45° field moving with a motion allowing it to cover thistotal field of 60°. The choice of the technology of the panoramic system1 is not limited: it can be passive but it is possible to generalize toan active system as long as the latter makes it possible to implementthe step of multi-plane fusion presented hereinabove; this also includeshyper-large-field optics exceeding 360°×180° or complete-sphere optics(for example 2 sensors with back-to-back fisheye optic exploring thecomplete sphere of observability). This panoramic system can alsocomprise a set of mutually non-independent optical sensors, togethercovering a maximum or a determined panoramic angular coverage, forexample identical from one image to the next. The set of these opticalsensors may not be overlapping, that is to say the global image obtainedat an instant by this set is not continuous (may comprise holes), the“holes” being filled in during the displacement of this set. Anexemplary 2D panoramic image obtained with an optic of fisheye type, andsectors (5 in this example) is shown in FIG. 5.

The trajectory may be computed in tandem with the displacement of thepanoramic system by location means measuring the relative displacementsof position and of attitude of the panoramic system in the scene such asGPS 12, inertial platform 13 or the like. This displacement can becontrolled by an operator via a man-machine interface 22 or beautonomous. The images thus obtained are such that the image of at leastone point of the scene is in at least 3 panoramic images respectivelyobtained according to various panoramic system-point of the scenedirections.

The step of processing these 2D panoramic images respectively obtainedat successive instants, by the processing unit 21, comprises thefollowing sub-steps described in conjunction with FIG. 2.

Step a) Determining at least one reconstruction plane (and preferablyseveral) in the scene.

Various reconstruction planes Cj can be chosen so as to establish the 3Dreconstructions by bringing to the fore various aspects of the scene,for example to cover the scene over a wide spatial and angular extent,or which will make it possible to have a better representation of thelatter. They are chosen freely by the operator or can be determinedautomatically as a function of the trajectory of the panoramic system,typically around the average of this trajectory computed between twosnapshots, and as a function of the complexity of the scene.

In the total absence of an initial 3D reconstruction and in theinitialization phase (=1st iteration), by default, the chosenreconstruction planes may be for example the 3 or 4 planes tangent to acylinder which would surround the mean trajectory of the system, so asto ensure a reconstruction in the various directions visible by thepanoramic system. For example, for a horizontal trajectory situated 100m from the ground, it would be possible to choose the followingreconstruction planes: the plane of the ground, a plane perpendicular tothe ground and tangent on one side to the cylinder surrounding thetrajectory, a plane perpendicular to the ground and tangent on the otherside of the cylinder, a plane parallel to the ground situated at aheight of greater than 100 m. Once an initial 3D reconstruction beginsto be constructed, these previously defined reconstruction planes can beupdated so as to approach or merge with the plane surfaces of thereconstruction in progress that are automatically or experimentallyextractable by an operator. When a single reconstruction plane does notsuffice to give a sufficient 3D representation of the scene, severalparallel or perpendicular planes are used to restore the uniqueness andthe completeness of the 3D representation. This is the case for examplewhen the scene comprises a concave object, or in the case where a singlereconstruction plane provides various measurements of 3D magnitudesdependent on the angle at which the measurement is made, and isconsequently incapable of providing a unique measurement, as illustratedin FIG. 4. This figure illustrates the Z-wise reconstruction ambiguityfor the point (X,Y): the acquisitions at the positions 1 and 2 of thetrajectory reconstruct z1 on the reconstruction plane P1, but theacquisitions at the positions 2 and 3 of the trajectory reconstruct z2on the same projection plane P1. A new reconstruction plane P2 is thenchosen to remove the ambiguity since we will have z1 for P1 and z2 forP2. A plane P3 is also chosen to find the lateral limits of the concaveobject.

In tandem with the displacement of the panoramic system, when new planesare revealed or disappear in the scene, it may turn out to also benecessary to renew the chosen reconstruction planes.

Step b): A concept of generalized rectification is introduced so as tobe able to rectify two successive 2D panoramic images according to anarbitrary direction. These two panoramic images are respectivelyacquired at an instant t and instant t+Δt and form a pair of panoramicimages.

This rectification consists in computing at least one projection planewhich is most suitable for the rectification and in applying thetransformation which transforms any sector of each of the two 2Dpanoramic images on each plane.

Each projection plane serving for the rectification, a so-calledrectification plane, can be chosen freely by the operator from among aninfinite choice of positions and orientations all parallel to thetrajectory of the panoramic system; the plane or each of them isindependent of the evolution of the line of sight of the panoramicsystem (which can pivot on itself in the course of its displacementalong its trajectory), in contradistinction to conventional stereoscopywhere the rectification plane chosen depends on the evolution of theline of sight and where the choices of rectification planes are verylimited.

An example of rectification planes referenced R1, R2, R3 is shown inFIG. 7; they are parallel to the trajectory. Also indicated is thedirection of the LdV (LdV1, LdV2) of the panoramic sensor at two pointsof the trajectory, which illustrates the fact that the choice of theseplanes is independent of the LdV.

Examples of rectification and reconstruction planes are shown in FIG. 8which is a view from above of a scene comprising 3D objects. On thetrajectory are indicated position pairs (1, 2, 3) of the panoramicsensor corresponding to 3 pairs of panoramic images acquired during thisstep b); with each position pair are associated two rectification planes(R11, R21 for pair 1, R12, R22 for pair 2 and R13, R23 for pair 3).Three reconstruction planes P1, P2, P3 have been chosen in the exampleof this figure.

In order to optimize the 3D reconstruction, each chosen rectificationplane corresponds to the various reconstruction planes. Therectification plane is for example chosen so as to be the closest (inthe geometric sense) to the reconstruction plane determined in step a).

The transformation which passes from the image to the rectificationplane is direct, that is to say does not make it necessary to passthrough an intermediate step of straightening in a focal plane as inconventional stereovision. This makes it possible to obtain a rectifiedimage which is:

-   -   independent of the rotation motion of the sensor and    -   without holes in contradistinction to what may be found in        conventional rectification,    -   more accurate since it is obtained through a floating direct        computation with no intermediate quantized image.

The mathematical steps of this rectification for a panoramic imageobtained at the instant t are, in the case of a sensor of fisheye type,the following:

Choosing of a rectification plane of a frame ({right arrow over(X)}_(i), {right arrow over (Y)}_(i), {right arrow over (Z)}_(i))associated with this plane P_(i), and of a sector of the panoramic image(possibly up to as much as the complete panoramic image if the field ofthe latter is included in the chosen zone on the rectification plane) tobe projected onto this rectification plane, this sector advantageouslymaking it possible to cover the rectification plane to the maximum. Ifthe projected sector of the image does not cover the whole of thepanoramic image, the sectors remaining in the image are projected intoother rectification planes, as in the example of FIG. 6 where the sector1 is projected onto a horizontal rectification plane and does not coverthe whole of the image to preserve a certain resolution; other verticalrectification planes are necessary to project the other sectors of thepanoramic image.

Computation of the transformation which transforms a point (x,y) of thepanoramic image into a point (X_(i), Y_(i)) of the plane P_(i); use ismade for this purpose of the correspondence which exists between theangular direction (θ, φ) of a point of the scene and of the coordinate(x,y) of the corresponding point in the image which depends on thepanoramic system chosen. In the case of a rectilinear panoramic system,this relation can be written simply:

If R is the radius of the position of the point (x,y) with respect tothe optical center, we have:

-   -   tgθ=(y−yc)/(x−xc) where (xc, yc) are the coordinates of the        optical center φ=k.R with k=rectilinear factor of the sensor

It thereafter suffices to write the equation of the plane P_(i) as afunction of the (θφ) found.

For a plane P_(i) whose normal is oriented according to (θ_(i), φ_(i)),with the focal plane for θ_(i)=φ_(i)=0 as particular case, it may bedemonstrated that the following relation holds for the particular caseof the centered projection, f being the focal length of the panoramicsystem:

$X_{i} = {f\frac{{\sin \; {\phi cos}\; \phi_{i}{\cos ( {\theta - \theta_{i}} )}} - {\cos \; {\phi sin\phi}_{i}}}{{\sin \; {\phi sin\phi}_{i}{\cos ( {\theta - \theta_{i}} )}} + {\cos \; {\phi cos\phi}_{i}}}}$$Y_{i} = {f\frac{\sin \; {{\phi sin}( {\theta - \theta_{i}} )}}{{\sin \; {\phi sin\phi}_{i}{\cos ( {\theta - \theta_{i}} )}} + {\cos \; {\phi cos\phi}_{i}}}}$

This transformation is an exemplary transformation in the case of arectilinear panoramic optic (fisheye type); it does not comprise thedistortion parameters of the panoramic system which can be computed andcompensated elsewhere. The transformation can readily be generalized andadapted to suit any panoramic system having its own optical formula.

It follows that for any point (x,y) of the sector of the panoramicimage, it is possible to find its corresponding rectified point in thechosen rectification plane and thus construct the rectified image inthis plane.

The various rectification planes chosen in the course of the iterationsand the above relation make it possible to define a sectorialrectification on the various rectification planes. A sector of thepanoramic image corresponds to an equivalent portion projected onto arectification plane. The sectorial decomposition of the panoramic imagedepends on the chosen rectification planes and on the footprint of theprojection on these planes.

Examples of rectified images are shown in FIG. 6. The first results fromthe projection of the sector 1 of the image of FIG. 5 onto a verticalrectification plane, the second results from the projection of thesector 2 of the image of FIG. 5 onto another vertical rectificationplane, the third results from the projection of the sector 3 onto adifferent vertical rectification plane from the first two, the fourthresults from the projection of the sector 5 onto a horizontalrectification plane.

This projection is repeated in the same rectification plane P_(i) for asector of another 2D panoramic image obtained at the instant t+Δt toobtain another rectified image, Δt being predetermined experimentally ordetermined in such a way that the displacement Dc of the system betweent and t+Δt produces a sufficiently large stereo base to be compatiblewith the accuracy desired for the 3D reconstruction. In the case forexample of an overflight at an average distance H from the scene, andassuming for example that minimum disparities of ⅛ pixel (current value)can be measured by the sensor 14, the displacement Dc to obtain thereconstruction accuracy sought d_(H) is: Dc=(resol/8)*H²/d_(H), whereresol is the resolution of the sensor 14 (for example 3 mrd for a 1Mpixel sensor furnished with a fisheye).

In the example cited, and assuming that the reconstruction accuracysought d_(H) is 20 cm for H=50 m, Dc must at least be equal to 5 m,thereby corresponding to an angular difference of 6° minimum between 2acquisitions by the panoramic system.

For the same accuracy d_(H) and for H=100 m, Dc must at least be equalto 19 m, thereby corresponding to an angular difference of 11° minimumbetween 2 acquisitions by the panoramic system.

The use of a panoramic system makes it possible to increase thereconstruction accuracy by increasing the distance Dc and the angularseparation between two acquisitions, beyond what can be done by a small-or mean-field sensor for one and the same spatial coverage of 3Dreconstruction. The stereoscopic base Dc serving for the 3Dreconstruction can be larger than that of a conventional stereoscopicmethod on account of the use of a panoramic field (and of the longerpresence of the objects in this field), and this allows the method agreater ultimate reconstruction accuracy, which accuracy is alsoincreased by the fusion of the measurements that the method offers.

By taking the above example of an overflight at a mean distance of 100 mfrom the scene (ground reconstruction over a field of at least 120°corresponding to a restored band of at least 350 m wide without countingthe reconstruction on the sides), the theoretical reconstructionaccuracy d_(H) becomes 10 cm for Dc=38 m and an angular difference of21°, and 2 cm for Dc=200 m and an angular separation of 60°; it ispreferably necessary to take account of the uncertainties ofmeasurements on the relative location between the viewpoints to obtainreal d_(H).

If we take the context of a visual inspection made by a panoramic systemwith a 1 Mpixel fisheye camera, at a distance H=20 cm from the scene,and assuming a displacement Dc of 10 cm between two acquisitions, thendetails of 15 μm in height or in depth can be restored (d_(H)=15 μm).

In order to average the various 3D reconstructions obtained during theiterations, and to thus benefit from a significant reduction in theerrors and the restitution noise, the real acquisition of the panoramicsystem can be faster while preserving the displacement between the pairsof 2D rectified images serving to reconstruct the 3D of the scene. Themethod then consists in taking a first pair of 2D panoramic images onthe basis of a displacement Dc, in doing an intermediate 3Dreconstruction with this pair, and then in taking another pair of 2Dimages again on the basis of a displacement Dc at the followingacquisition so as to redo an intermediate 3D reconstruction, doing sofor as long as the scene points concerned in these various pairs ofimages remain in the field of the panoramic system.

Step c): The stereoscopic pair of rectified images in the plane P_(i) isutilized to define an intermediate 3D reconstruction in a frame relatingto this P_(i).

The intermediate 3D reconstruction in a 3D frame tied to the P_(i), theso-called intermediate 3D frame, is obtained by matching point-to-pointthe two rectified images in P_(i), aided by the knowledge of the motionof the panoramic system. This matching is a dense process, whichmatches, in so far as possible, each of the points of a 2D image of thestereoscopic pair with a point of the other image. It can be carried outby a more or less hierarchized local correlation process and can beaided by matchings carried out at t-Δt or t-NΔt, N being an integer >1;the large-field nature of the panoramic system and the very possibilityof viewing the same scene from a different angle, something which asmall-field system used traditionally in stereoscopy does not allow,makes it possible here to remove certain occultations or ambiguities.

Step d): transforming this intermediate 3D reconstruction into a fixed(=absolute) 3D frame including the reconstruction plane (or planes)determined in step a), the so-called 3D reconstruction frame. Atransformed intermediate 3D reconstruction is thus obtained.

Step e): repeating steps b) to d) at least once on the basis of a newpair of panoramic images (this may be a new image pair formed on thebasis of previous images, or this new pair results from a newacquisition coupled with one of the previously acquired images) and ofat least one other rectification plane P′_(i), to obtain at least oneother transformed intermediate 3D reconstruction; the same 3Dreconstruction frame as in step d) is kept. These iterations can besuccessive in the sense that steps b) to d) are strung togethersuccessively in this order; these iterations can also be carried out inparallel (several steps b) are carried out in parallel with severalrectification planes P_(i) determined in parallel, etc).

Preferably, these steps b) to d) are repeated as long as at least onereconstructed scene point remains in the field of vision of thepanoramic system.

Step f): The transformed intermediate 3D reconstructions are temporallyfused by a specific fusion method which utilizes the spatial andtemporal redundancies of the intermediate reconstructions. This isobtained by temporally fusing at least two transformed intermediate 3Dreconstructions obtained in the 3D reconstruction frame, to obtain acorresponding 3D reconstruction.

Step g): repeating steps b) to f) for each reconstruction plane chosenin a), with the same panoramic images but with different sectors, tothus obtain as many 3D reconstructions as chosen reconstruction planes.These 3D reconstructions or the intermediate 3D reconstructions obtainedin the course of these iterations are advantageously fused spatially toupdate the final 3D reconstruction or reconstructions, and thus increaseaccuracy and robustness of these reconstructions. The spatial fusion ofthe 3D reconstructions constructed according to various planes takesaccount of the accuracy of reconstruction of the various elements ofeach reconstruction which is not the same according to the variousplanes and that can be predicted mathematically. This spatial fusion isobtained by utilizing several rectification planes corresponding to thevarious sectors of each image used.

The set of steps a) to g) are also preferably repeated at least oncewith new pairs of panoramic images, for example with intermediate imagestemporally shifted from the previous ones, or with other sectors of thealready considered panoramic images. This makes it possible to have acontinuous process of updating the final 3D reconstructions. These newpairs of panoramic images may originate from each panoramic imageacquisition but not necessarily.

Here again, these iterations can be conducted successively or inparallel.

The utilization of the redundancies and of the quality of the 2Drectified images (quality defined for example by the angular disparityexisting between the rectification plane and the reconstruction plane,or else by a confidence coefficient of the matching that led to eachintermediate 3D reconstruction) allows the method to produce aconfidence map conveying the quality of the final reconstruction. Thisconfidence map is constructed pixel by pixel for each 3D reconstruction,by considering the number of times that each pixel has been constructedand whether the conditions of this construction were good, these beingdefined for example as a function of an experimentally or mathematicallydetermined threshold of matching quality. Also considered are the caseswhere several 3D magnitudes are obtained for one and the same pixel as afunction of the angle of observation, in which case additionalrectification and reconstruction planes are created to remove theambiguity, for example for concave objects which require more than onereconstruction plane in order to be reconstructed correctly, as in theexample of FIG. 4.

We now consider the mosaicing of the 2D images of the scene, thecomposition of these 2D images forming a global image called a mosaic.This mosaic generally comprises several 2D textured planes present inthe 3D scene or which approximate it, but may also be on a 3D surface.

The utilization of the 3D reconstruction of the scene createdprogressively makes it possible to project each 2D image originatingfrom the sectorial decomposition of the 2D panoramic image onto variousprojection planes (or surfaces) also called mosaicing planes. Theseprojection surfaces are the surfaces on which the mosaicing isconstructed; they may be chosen freely by the operator or may bedetermined automatically on the basis of the 3D reconstruction. Asindicated hereinabove, some of these surfaces may be warped (curved) oreven be a 3D surface the modeling of which is known.

In the case of a panoramic system viewing a highly 3D scene exhibitingvarious faces, several mosaicing planes (or surfaces) can (andbeneficially may) be chosen. By highly 3D scene is meant a scenecontaining many 3D elements producing significant disparities betweentwo successive acquisitions, as is the case for example for a droneoverflying an urban setting at low flight height. The method ofmosaicing utilizes the fact that the projection surfaces or planes havedifferent orientations so as to best project the textures of the imagesonto each of the projection surfaces or planes. It is recalled that thetexture is a set of intensities of pixels over an image region.

The utilization of the 3D reconstruction makes it possible to alsopreserve only the visible parts of the projected images. This makes itpossible to avoid projecting onto a mosaicing plane portions of imageswhich would belong to other portions of the scene.

The multi-plane (or multi-surface) projection mosaicing process ispreferably repeated at each new 2D image acquisition performed by thepanoramic system, and the new mosaic is fused with the old one (obtainedat t-1) so as to update the latter.

The result of these various projections and of the continuous fusion ofthe mosaics is a conformal image (that is to say with no geometricdeformations) that is very extended over each projection plane. Thisresults directly from the fact that the method of 3D mosaicing,explained hereinbelow in detail and described in conjunction with FIG.3, simultaneously computes the 3D reconstruction of the scene and theprojections of the textures on it, that the method eliminates the hiddenparts or the poorly resolved parts in the projection and that thismethod is repeated in all directions and following the whole of thetrajectory.

The 3D reconstruction of the scene and the projections of the textureson it are computed at each so-called initial 2D image acquisition, anacquisition being separated from the previous one by a time interval Δtdefined above.

According to an alternative, the 3D reconstruction of the scene and theprojections of the textures on it are computed at each image acquisitionof the panoramic system and at high frequency on the basis of previousimages stored earlier. More precisely: the intermediate images lyingbetween two successive images separated by Δt serving for the 3Dreconstruction are stored in such a way as to be able to also be usedfor the 3D reconstruction in the manner of a FIFO, the acronym standingfor “First In First Out” (each new image acquired is compared with thefirst image stored so as to establish a new instance of 3Dreconstruction, this first image is thereafter erased from the list andthe last one added to the updated list). Moreover, the intermediateimages may also serve to facilitate the correspondence between the firstand last image, or serve to fill “holes” in the 3D model.

A mosaic is then obtained on completion of the following steps A) to E)described in conjunction with FIG. 3, for each new 2D image acquired bythe panoramic system.

According to a first embodiment, the 3D reconstruction and the mosaicingare performed in a successive manner after each acquisition; thisassumes that a new 3D reference reconstruction has just been performedsubsequent to (one or more) 3D reconstructions already performed.

According to a second embodiment, the 3D reconstruction and themosaicing are performed in parallel after each acquisition; this assumesthat the mosaicing is performed whilst a new 3D reconstruction is stillin progress, in which case the 3D reference reconstruction is thatperformed at one of the previous acquisitions of 2D images, or indeed a3D reconstruction performed previously.

These various steps will be described in greater detail.

A) Choosing 3D projection planes (or surfaces).

This first step consists in choosing the 3D projection planes orsurfaces on which the mosaic is constructed. These 3D projection planesor surfaces can be chosen freely by the operator at a given moment ofthe mosaicing or computed automatically on the basis of the current (orreference) 3D reconstruction of the scene according to predeterminedcriteria (for example planes parallel to the reconstructed surface ofthe ground or principal planes extracted from the 3D reconstruction). 3Dprojection surfaces may also be used if the scene lends itself theretoand if the operator sees a benefit therein; this makes it possible forexample to represent objects of the scene or a scene background whichhave particular geometric shapes, but this in no way detracts from theconformity that could be obtained by multiple projections that would beexclusively plane.

B) Determining rectified images or associated sectors in the panoramicimage.

When rectified images have been computed during one or more previous 3Dreconstructions, the closest (in the geometric sense) to the projectionplane is chosen and the parameters of the direct projection (with nointermediate step) onto the projection plane are computed.

If this rectified image is too far away, that is to say is not close tothe projection plane with respect to a threshold predetermined by theoperator for example, a 2D panoramic image, the so-called current 2Dpanoramic image, is acquired and the parameters of the direct projectionof this current 2D panoramic image onto the projection plane and thesectors of this current image which will be used during the directprojection of step D are computed.

In the two typical cases, the projection is not performed straight awaybut the projection parameters are placed in memory in order to be usedby step D).

C) Determining the utilizable parts (that is to say that have sufficientresolution) that can be used for the projection of step D, with the aidof the 3D reconstruction to compute the hidden parts in the projectionplane on the basis:

of the 2D rectified image if the first case of step B applies,

or of the sectors of the current panoramic image which will be used inthe direct projection, if the second case of step B applies.

The 3D reconstruction that has just been computed makes it possible toautomatically compute the hidden or weakly resolved parts (andconversely the utilizable visible parts) in the projection plane whichwould result from maskings present in the scene. This amounts toselecting the textures to be preserved in the mosaic. This computationis accurate because the 3D reconstruction has been constructed firstlyin the frame tied to the panoramic system.

One of the particular features of the 3D mosaicing according to theinvention is to profit from the computation of the hidden parts so as toeliminate on the various projection planes the masks generated by thescene and to consider on these planes only the visible parts. This makesit possible to temporally mosaic only parts of scenes which are alwaysvisible and thus to avoid deformations due to the projection of parts ofthe scene not belonging to the projection plane (defect present in aconventional mosaicing which from the outset projects the whole of theimage onto the projection plane without being able to take account ofthe maskings by the elements of the scene evolving in tandem with thedisplacement of the sensor).

D) Projecting the textures selected in the previous step onto the 3Dprojection planes or more generally onto the 3D projection surfaces andfusing the textures in each 3D projection plane or surface so as to thusobtain a conformal mosaic on several planes.

The textures selected in the previous step are projected onto theprojection planes (or surfaces), and fused temporally with the currentmosaic to form a new mosaic.

It is important to note that, by projecting onto surfaces or planesarising from a 3D reconstruction built on the basis of the same baseimages as those serving to project the textures, very high accuracy inprojection (and in the geometric transformations between image frame andframe of the reconstructed scene) is made possible. It is also thiswhich ensures the conformity of the mosaicing produced. But theprincipal element of conformity results from the fact that the mosaicingutilizes a 3D reconstruction carried out in the same frame as the mosaicand uses only the portions of images that are not masked in its mosaicprocess. In the case where an external 3D reconstruction were used whichdid not arise from the base images serving for the projection, therewould necessarily be uncertainties in the relative position of thesensor in relation to the scene and in the overlap of the projection.

E) Presenting the mosaic to the operator according to various planes ormore generally according to various 3D presentation surfaces, byprojecting the textured 3D reconstruction (or reconstructions) ontothese presentation planes. These presentation planes are chosen freelyby the operator and serve solely to present the results of the mosaicaccording to various perspectives chosen by the operator. The mosaic canbe presented to the operator presentation plane by presentation plane,or according to the planes representing the unfurling of the curvedsurfaces onto which the textures have been projected (in the case of aprojection onto a cylinder for example). The textured 3D result canobviously also be presented directly in 3D virtual form using suitablesoftware. The projection result provides an ever conformal image, thisnot necessarily being the case, as has been explained, with aconventional mosaicing method.

This omnidirectional simultaneous 3D reconstruction and mosaicing methodis not limited to an optical panoramic system. It is very possible toutilize the textures measured over a large directional field by a meansother than optical, for example by an active means of lidar or sonartype; the method could then also utilize the distances given by theinstruments.

Among industrial applications may be envisaged:

-   -   the real-time 3D and textural restitution of a scene overflown        by a drone or an aircraft (application to the production of 3D        maps, of orthophotographs, application to surveillance, etc.),    -   aid to terrestrial or airborne navigation,    -   industrial, medical or other visual inspection.

1. A real-time method of 3D reconstruction of a scene, without a prioriinformation about the scene, by means of 2D panoramic images of thescene, which comprises a step of processing these 2D panoramic images,characterized in that the 2D panoramic images arise from a panoramicsensor moving in displacement over an unconstrained 3D trajectory, suchthat the image of at least one point of the scene is in at least 3successive 2D panoramic images obtained according to various panoramicsystem-point of the scene directions, and in that the step of processingthese 2D successive panoramic images respectively obtained at successiveinstants comprises the following sub-steps: a) determining or updating,at each new acquisition, reconstruction planes of different directionsin the scene to be reconstructed, b) determining, on the basis of pairsof panoramic images sliced up into sectors covering in totality each ofthe panoramic images, respectively formed of a 2D panoramic imageobtained at an instant t and of another 2D panoramic image obtained atan instant t+Δt and for each chosen pair of images, rectification planescorresponding to the various chosen reconstruction planes, determiningthe various sectors in the two panoramic images corresponding to thevarious reconstruction planes, and projecting onto each of therectification planes the corresponding sector of a 2D panoramic image ofthe pair, in a direct manner, that is to say by applying a mathematicaltransformation which causes each pixel of each sector of the panoramicimage to pass directly into a corresponding pixel in the correspondingrectification plane without passing through an intermediate step ofprojection onto focal plane, so as to obtain a 2D rectified image, andprojecting onto these same rectification plane and in a direct manner asector of the other 2D panoramic image of the pair so as to obtainanother 2D rectified image, c) matching, for each chosen rectificationplane, the two 2D rectified images so as to obtain an intermediate 3Dreconstruction in a 3D frame tied to the rectification plane, theso-called intermediate 3D frame, d) spatially fusing the intermediate 3Dreconstructions, e) transforming each fused intermediate 3Dreconstruction into a 3D frame including the reconstruction planeschosen in a), the so-called 3D reconstruction frame, so as to obtain atransformed intermediate 3D reconstruction, f) repeating steps b) to e)at least once on the basis of a new pair of 2D panoramic images and ofat least one other rectification plane, so as to obtain at least oneother transformed intermediate 3D reconstruction in this 3Dreconstruction frame, g) temporally fusing, for each chosen constructionplane, at least two transformed intermediate 3D reconstructions toobtain a 3D reconstruction of the scene, and h) steps b) to g) arecarried out for each reconstruction plane chosen in step a), with thesame panoramic images but with different sectors, so as to thus obtainas many 3D reconstructions as chosen reconstruction planes.
 2. Themethod of 3D reconstruction of a scene as claimed in claim 1, whereinsteps a) to h) are repeated at least once with new panoramic imagesdifferent from those already used, so as to thus obtain at least oneother 3D reconstruction of the scene.
 3. The method of 3D reconstructionof a scene as claimed in claim 1, wherein the 3D reconstructionsobtained according to different reconstruction planes are spatiallyfused.
 4. (canceled)
 5. The method of 3D reconstruction of a scene asclaimed in claim 1, wherein the reconstruction planes are chosen freelyby an operator.
 6. The method of 3D reconstruction of a scene as claimedin claim 1, wherein each reconstruction plane is determinedautomatically as a function of the trajectory.
 7. (canceled)
 8. Themethod of 3D reconstruction of a scene as claimed in claim 1, whereinthe trajectory is determined in tandem with the steps of 3Dreconstruction.
 9. The method of 3D reconstruction of a scene as claimedin claim 1, wherein each reconstruction plane is determined as afunction of the 3D reconstruction that has just been established and asa function of the scene scanned by the sensor.
 10. The method of 3Dreconstruction of a scene as claimed in claim 1, wherein eachrectification plane is determined as a function of the trajectory of thepanoramic system, and independently of the line of sight of thepanoramic system.
 11. The method of 3D reconstruction of a scene asclaimed in claim 1, wherein the panoramic system exhibiting a determinedpanoramic field, the set of the sectors associated with therectification planes covers the whole of this panoramic field.
 12. Themethod of 3D reconstruction of a scene as claimed in claim 1, wherein Δtis determined as a function of a required 3D reconstruction accuracy.13. The method of 3D reconstruction of a scene as claimed in claim 1,wherein the panoramic system exhibiting a determined panoramic field,the repetition of step e) is performed as long as the point of the sceneis in the panoramic field of the panoramic system.
 14. The method of 3Dreconstruction of a scene as claimed in claim 1, characterized in thatit comprises a step of producing a confidence map related to a number oftimes that a point of the scene is viewed and reconstructed.
 15. Themethod of 3D reconstruction of a scene as claimed in claim 1, whereinthe panoramic system comprises at least one optical sensor.
 16. Themethod of 3D reconstruction of a scene as claimed in claim 1, whereinthe panoramic system comprises at least one active sensor of lidar orsonar type.
 17. The method of 3D reconstruction of a scene as claimed inclaim 1, wherein the panoramic system comprises several sensors abletogether to reproduce a maximum panoramic angular coverage. 18.(canceled)
 19. The method of 3D reconstruction of a scene as claimed inclaim 1, wherein the panoramic system is immersed in the scene.
 20. Themethod of 3D reconstruction of a scene as claimed in claim 1, whereinthe concave objects of the scene having several 3D inclines according tothe angle of observation, rectification planes and additionalreconstruction planes are added so as to remove the ambiguity of theinclines.
 21. An apparatus for 3D reconstruction of a scene, whichcomprises a panoramic system able to form images of the scene, furnishedwith relative image to image location means and, linked to thispanoramic system, a computer comprising means for implementing the 3Dreconstruction method as claimed in claim 1, and image processing means.22. (canceled)
 23. The method of 3D reconstruction of a scene as claimedin claim 1, wherein the panoramic system covers the whole of the spaceand of the scene around the sensor.